The familiarity hypothesis: Explaining the behavior of deep open set methods

In many object recognition applications, the set of possible categories is an open set, and deployed system will encounter novel objects belonging to unseen during training. Detecting such “novel category” usually formulated as anomaly detection problem. Anomaly algorithms for feature-vector data identify anomalies outliers, but outlier has not worked well in deep learning. Instead, methods based on computed logits visual classifiers give state-of-the-art performance. This paper proposes Familiarity Hypothesis that these succeed because they are detecting absence familiar learned features rather than presence novelty. distinction important, familiarity-based fail situations where novelty present. For example when image contains both a one, familiarity score be high, so noticed. The reviews evidence from literature presents additional our own experiments provide strong support this hypothesis. concludes with discussion whether inevitable consequence representation